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ABSTRACT 

The purposes of this study were to examine the 
relationship of students and instructors! evaluations, to discover 
which elements of evaluation seem to best predict the grade awarded 
by each group, and to compare the dimensions of evaluative judgments 
for students and instructors. College students enrolled in a public 
speaking course at Indiana University served as subjects. The public 
speaking course was designed so that subjects ptesented speeches to 
classes with siici *” vere unfamiliar. On days when students were 
not scheduled to speak, aey remained in their regular class to serve 
as an audience. Each sp: .ker received a written critique and grade t 
from all the student auuience members as well as from a visiting 
instructor. A five point rating scale was used by all raters tc 
record the judgments of the speeches... Some of the results indicated 
that student audiences pay more attention to general effect ard 
adaptation than do instructors in assigning grades. Also, both 
students and instructors come to highly similar contlusions 
concerning student speeches, but evidently for different réasons. 
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Poth instruction ang learning in public speaking courses a ad 
heavily on the ability to evaluate sneech making, ; Teadheva Haed to 
“L ddentary effective speeches; students must have’ some means of 
assessing their Prosress, Whavever the definition of success, cne 
fact is clear: no speatne dagceeds unless an audience thinks so. 
sfeech success depends on effectiveness with an audience; no public 
* sveakers to our tnowledce, ever swayed masses in an empty room. “ 
Even ccnsumnate pragtitioners of oral communication are sometimes 
denied success--witness Clarence Darrow in Hawaii. Since success is 
an audience- centered variable, it is worthwhile to ask on what beagle 
audiences evaluate aipashee, This and related questions form the 
central focus*of the present paper. « 
the disturbinr element about classroom speech evaluation is that 
we seem to lose touch of the concept of "audience" when designing 
rating RignS Althouch we teach audience analysis and ahavtation, 
our judements of speech effectiveness have most often been made in 
terms of an audience of’ one: the instructor. Even when evaluations 
have been cathered from more heterogenous audiences, their worth 
has been discussed in terms of agreement with an suseruetor!s rating. 
However, an important question remains unanswered, anes are 
more students present in the classroom than there are inaGeaniore: 
True audiehce, adaptation by the avesicee must take these listeners 
into account. ; If we believe that public speaking is coumoninarien. 
au 4s appropriate to ask if the ratinrs of instructors accurately 


reflect the evaluation of the audience to whom the sveech may be 
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delivered, In HERES OPae) are there alae aint and definable.’ 


* 
relationships between the un niend made of a public “speech by each 


of the two audiences--the instructor gnd the siudenvay - : 
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Previous Research ; ‘<* © ss . ' 
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Several previous investigations have. examined the ayationete 
between instructor and student evaluation of pubtig speeches. cne ~ 


of the earliest of these was done by ronrce,-et. 1. (1935) 
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. re a 
comparing eae Sree grades and’ gtudent retines of the seme 


speeches, The total, picture of student-and instructor evalations 
: . : f 


led ‘these..researchers to conclude: ° . os hs 
"instructors! rrades. and students’ ratings agree fairly. 
closely .as to the relative effectiveness of different —' 
speakers, and to that extent students! ratings may be ; # 
‘said to be valid measures of judged effectiveness of : 
the same type recognized as such by speech ileal 
(pp. 25-26) 


In short, the sinsaicia was "that the students and the , 


: sy ' . & 
instructors, using the sane basis for judgments, arrive at highly 


similar conélusions concerning shal NE: ‘ Oe id 
} In an unpublished Master's thesis, Bow (1959) compared the 


srade awarded to student speakers by ercups’ of LeanuERe (8 3) 

and students (N ws 10). Bxaminine the Be PEO EOREND aN tha end of 

two separate semesters of a public speaking course, ne observed 
correlations were found to be .56 and 61. ' “yiseman arid Parker (1965) 
found no airrevencce between composite wietrustor ratings and the 
average of five composite student ratings. However, they did find 


that--overall--students tend to award higher grades than instructors. 
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roe Te . Finally, Shepard (19703 aiscovpred that evetpaet students evaluate «i sie 
‘speakers into ie” ‘same rank-order as ES Anstructors. AL a Peet 
In sun, tne\gesearch evidence seoms. to point out eRe while; 
students ‘nay grade a bit higher than do instructors, the relative 
eS siixe of those graded is highly consistent with the order perceived ™ 
by instructors. ees a, in all thee idee, we atild do not 
know whethe? the ‘basis upon’-which the students grade,is the same as 
itis for instructor's 3; Rrades. Fonroe's assertion (1935). remains’ 
an assertion. -Purther, we do OT. mua to what extent ‘aa! ceipenanee. 
: or elenents of SeAeaveen predict? the find grades awarded, 
M at in sims of these issues, the “purposes of one study are 
" threefold: “(1) ta. examine’ the relationship of etudente! and 
ypstructore! byatuations (2) _to discover ad elements of evaluatior ‘Zs 
seem to best svediet the grade anardch by each nroup, and (3) to ¥ 


7 compare the dimensions of evaiuative judgments for students.and 
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instructors, - : 
‘ Procedure 


A The public speaking course at Indiana University is a two-hour 
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semester course designed to ‘gupeys each students' ability to address 
an audience, Gften the student »econes so thoroughly familiar with 
the other members of the‘class that audience adaptation, at least - 

near Riis end ate the course, becomes almost a reflex. - Consequently, 

at least one opportunity is provided for enon Scueeue to speak to an 
‘unfamiliar avdiaise, Dering the last bis ‘weeks of: the course,, wns 
final “speech ‘is. delivered to, and evaluated by, a cltss other ‘ 
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hes ~~ he 
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than the one normally attended by the* stydent. From the perspective 
of the student, the regular cl&ss is exchanged for another class 
j - ’ ‘ . ran 
and instructor. 
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The "exchange" speech is accomplished by having student speakers 


. 


from one class--usually 5 at a time--go to another section of the 

i ¢ 
same. course meeting at the Sane day ,and hour. Simyltaneeusly, 5 of 
the 20 students in, the other class are coming to visit .the first 


class. Thus, from *he student epeaker's viewpoint, the audience is 
“@ 


comprised \of the 15 or so students in the Le eaaca elase plus the new’ 


ee 


instructor. On the days when students are not scheduled to speak, 


‘they remain’in their regular class to serve ¢ as an audience, Each 


student BpSRMR receives a written critique and grade from all the 


student audience members as well us from the: iD ca instructor. 


aw 


Usually, ‘the Grade. for this “speech i some eninbinehien of student 


and inatructer awarded grades. It should be enphasized that the 


f exthange speech is a regular and quite well known feature of the. 


course, ‘When students in this caurse make speech evaluations, they 


e r 


. are performing an expected and meaningful element of the course. K 


Tas, any element. of the spengeanory” atmosphere which occasionally 
contaminates Lioeneae abioea. a may be discounted in this etudy. 
The. course *itself, in the Fall semester of 1973, was taught by 
one Ph. D. (in this case the senior author) and by 17 graduate 
student Associaté Instructors, all but 3 of whom held the Master's 
degree in Speech, Fach of the 48 seations of the course had»an 


: fs 
enrollment of from 18 to’ 20. students. 
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form to record their judgments of the: speeches. fhe form wag 


in echeduling the excharigel speeckes all students were ‘randomly 


assigned both Ne ans aay DF presentation and to speaking order. on ‘ 


the day mes hey epnke, The: sample je Rpeeon evaluations for - 


this aug was drawn oyily from the aap hd of the eset speeche: 


Sampling this one day reduced the nossibjJe number of evaluations 


but it. aaeuned hee, student evaluators’ had ‘at’ least one duy eee ; “4 
experience with the evaluation FORE sii with the general toma’ 
of the assignment itself. ? ( wer < 


Ail instructors and student evaluators used a common rating, 


composed of a 5=point Likert type scale of the following items: 
"Thesis," "adjustment to Audience," ‘aumenetehe Material, 4 


"organization," "Presentation," ane "General Effect." In addition, 


a letter grade was anarded to, the speaker ve all students and the 


instructor. These 7 evaluations: formed the "basis for the wine 
* . cay 
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Results and Discussion: ; gk 
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The first concern was to examine the relationship between 


the instructor awarded grades ant the student awarded grades. — # 


Usable data.for 70 such comparisons was ‘available. Since, in 


some instances, the grades awarded by students were’ tadly skewed, 
the median class grade- vas .compared with the Anstruotor's srade,. ~ 
The correlation between thease rrades was 56 (p'< 6%; (This ( : 
correlation is eokavkaply close to those found by Bowerg (1959) a 7 i 
--,56 and 61,3 Our finding strongly supports the fact that // | 


instmictors! grades and students! grades covary together. 
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The meer concern was to determine which of the si: wenden oF 
evaluation would best predict awarded grades by ‘students and by / 
instructors. To accomplish this, two stepwise regressions were - 
performed, It was decided not to retain any variable in the- 


resression unless it added at, least 1% explained variance to the 
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t * z oe 
solution. Using’ this oriterta the. results ard shown ir. Table 1. ° so 
ia i i es i rene a mt ee & 
o 
‘| ‘Insert Table 1 about here. 
SS re en Re RT Pe en gE ee ee i‘ « a . } 
4 Keeping in mind that the two problems were solved in precisgly WA 


the same marner and the variables entered in the same order, the two- 
solutions aiffer substantially. General BE Tect is the most 
explanatory variate in the student sedutzons it “does not even ‘ 


appear in the instructor solution. Crganization adds. not explained 


€ 


variance for instructors; at is totally absent for students. 


Whatever is happening in the evaluative process, it spree to be 


_ very different for instructors and for atudenas” ae ig <3 


7 


, This result prompted us to examine the respective factor . 


“structure for students and instructors, (A standard orthogonal. 


rotation was chosen; Nultiple R® was inserted in the diagonals. ), 2 tg 
The results are shown in Table 2. . 
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Insert Table 2 about here. 
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In both cases, the. result was a clear one factor solution. 
¢ “e oGiak ; i 
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: Thig finding partially corroboretes Secker's (1962) findings. He 
Relea 3 factors ‘but they all appeared to be measuring one re 
concept. 2 In the present study, each group seems to use the scales 
as measures of "Speaking Effectiyeness." However, it is worth 
noting that the factor loadings ditrer for students and’ instructors 
indicating the items differ in importence for exch group, If we 
- examine the pave order correlations between factor loadings for 
: both groups, Spearman's rho reveats there is no systematic 7 
relationship between the ‘tris’ solutions (rg = .5iks pis ‘not : 
Significant), We ‘ean only conclude that there is no relationship 
between the rélative importance of the same scale items for students . 
and SEER OUEEES 
Finally, our concern was to compare the dimensions of 
evaluation by atndente and instructors, Desiring to find those 
variables which maximally distinguishedbetween the two groups, 
factorYal discriminant analysis was the tool chcsen (Cooley and 
Lohnes,' 1971). _ All six scales plus awarded grades were entered | 
into the problem, \ Mable 3 shows the results. 
' | a \ 
. Ps | Insert Table 3 about here, ‘ wt og 
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All of thé-seven variables figure in the discriminant function. 
Ie is’ interesting that one of the least important of these is.the 
awarded srade,. The most important 1s the way in which the scale 


of organization is used by the two groups.’ - The other scales fall 
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should be used for its own best function. 
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BOMSEIASS in atten in. hort, isth students, and, apECEREsn es 


come to highly similar, conclusions concerning student spedches—~ ; 


bat for different reasons. i : oe lt 


Implications § — _ ; 

our discussion of the students in. the lass &8 the eudience, 
snould not be alee to mene that instructors”. vatings are of no - 
value, The Syeawahtane Seryr tee here sipeent that student 
audiences ‘pay more attention to general effect and adaptation than 
do instructazs on assigning wreden, Their ratings may be a niore 
accurate indteation of the speaker's "success" with the audience. 
Nonetheless, the ‘instructor's SARTRE EEN. which focuses on 
presentation and organization, may be of nore iaegaine in helping “the 
shane ¥0, lnniora in guise ateaking.. Each type of evaiuation P 
¥ 

en another front, we ease urge caution in adopting peer 
evaluation as. a substitute for a@ teacher in these days of vanishing’. 
budgets. Those who. might be tempted ie advocate peer evaluation 
because "you get the saue answers anyway," are “probably correstt< 
but for very airferent reasons than first suspected, Surely, one 


r 


study ad answer all ‘the questions concerning evaluation. 


‘However, ‘one finding seems rather clear, The evaluation of student 


. speakers by other students and. by inetedatens may be ‘very similar; 


1} a 
but, the answers are reached for quite different reasons. 
° : as | 
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Regression cf Six. Evaluative Scales», 


hehe a on Gredes Awe sded 
' o 
. Students (N = 456) i ? 

. 7 ; '  Ynorease in ' Tote#l Explained 
Variable x. R. Explained Variance Variance - , 
General Effect «fae? en . 536 ° 
Presehtation 796° 4097 . : 633 
Supporting Yaterial — 818 "035° fe : 668 
Adjustment to Audience .835__ «028. * 697° 

F o? a , _ , 
Thesis . 841 + 04% 708 
8 ‘3 ; . & a” ' 
a Instructors (N = 74) ; 
a: i 
Presentation 743 ---- 2552 
\ j ; 
Crganization | 2825 aleg J -580 
Supporting’ 7 aterial 847 8037 0717 » 
Thesis” a .023 : 7, 
. eo a \ 
_ ; 
{—~~ ” . 
* ais . 
' > 
; i 


Table 2 
Unrotatea Factor Solutions 
x for Six Evaluative Scales 


(Students and Instructcrs) 
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Variable Gra see) Ne ah 
Thesis -.628 0573 
Adjustment to Audience * =,704 641 
Supporting Material -.608 631 
Organization -.684 676 
Presentation ~.729 79 
General Effect -.765 6639 
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F , Teble 3 
Discriminant Analysis of Six Evaluative Scales . 


and Grades for Instructors and Students 
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° Discrimination 
Variable Tndege 
Grades ( 2 566 
Thesis 0395 
Adjustment to’ Audience 7H 
Supporting Faterial 2592 
Crganization 815 . : : 
Presentation 0738 
General Effect 688 


*These coefficients may be thought of as betas or as factor 
loadings in their function. They are measures of how’ 
important the veriable is in distineuishing between the 
two groups. see Cooley and Lohnes (1971) for further 
discussion, 
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